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1 Introduction 



it al. 



Recent advances in techniques for the formal analysis of neural networks (Amit 
" 1987j: pardncij , |1988| ; [Tsodyks and Fcigclman| , |1988| ; [Trevei ' 



199C; Nadal and 



Parga, 1993) have introduced the possibility of detailed quantitative analyses of 
real brain circuitry. This approach is particularly appropriate for regions such as 
the hippocampus, which show distinct structure and for which the microanatomy 
is relatively simple and well known. 

The hippocampus, as archicortex, is thought to pre-date phylogenetically the 
more complex neocortex, and certainly possesses a simplified version of the six- 
layered neocortical stratification. It is not of interest merely because of its simplic- 
ity, however: evidence from numerous experimental paradigms and species points 



lems of cognitive neuroscience (Bcoville and Milner, 


1957; 


Weiskrantz , 


1987 ; iGaffan 


1992; 


Cohen and Eichcnbaum, 1993; 


McNaughton and Morris, 


1987: Rolls, 


19911) 



Much useful research in neurophysiology and neuropsychology has been directed 
qualitatively, and even merely categorially, at understanding hippocampal func- 
tion. Awareness has dawned, however, that the analysis of quantitative aspects 
of hippocampal organisation is essential to an understanding of why evolutionary 
pressures have resulted in the mammalian hippocampal system being the way it 
is (Amaral et al., 199C; Treves et al., 199£; Btephan, [1983 ; Witter and Groenewe 



1992) 



Such an understanding will require a theoretical framework (or formal- 
ism) that is sufficiently powerful to yield quantitative expressions for meaningful 
parameters, that can be considered valid for the real hippocampus, is parsimonious 
with known physiology, and is simple enough to avoid being swamped by details 
that might obscure phenomena of real interest. 

The foundations of at least one such formalism were laid with the notion that the 
recurrent collateral connections of subregion CA3 of the hippocampus allow it to 
function as an autoassociative memory (Rolls, 1989, although many of the ideas go 



back to Man, 1971), and with subsequent quantitative analysis (reviewed in Treves 
md Rolls| , 1994). After the laying of foundations, it is important to begin erecting 
a structural framework. In this context, this refers to the modelling of further 



1 



features of the hippocampal system, in a parsimonious and simplistic way. Treves 



(1995) introduced a model of the Schaffer collaterals, the axonal projections which 
reach from the CA3 pyramidal cells into subregion CA1, forming a major part of 
the output from CA3 and of the input to CAl. The Schaffer collaterals can be seen 
clearly in Figure [l], a schematic drawing of the hippocampal formation. This paper 
introduced an information theoretic formalism similar to that of Nadal and Parga 
(1993) to the analysis. As will become apparent, this approach to network analysis 
appears to be particularly powerful, and is certain to find diverse application in the 
future. 

Once the rudiments of a structural framework have been erected, it is possible to 
begin to add to the fabric of the theory - to begin to consider the effect of additional 
details of biology that were not in themselves necessary to its structural basis. This 
is where the contribution of the work described in this chapter lies. The analysis 
described in (Treves, 199E ) assumed, for the purposes of simplicity of analysis, that 
the distribution of patterns of firing of CA3 pyramidal neurons was binary (and 
for one case ternary), although it considered threshold- linear (and thus analogue) 
model neurons. Here we shall consider in more detail the effect on information 
transmission of the possible graded nature of neuronal signalling. Another simple 
assumption made was that the pattern of convergence (the number of connections 
each CAl neuron receives from CA3 neurons) of the Schaffer collaterals was either 
uniform, or alternatively bi-layered. The real situation is slightly more complex, 
and a better approximation of it is considered here. 



2 A model of the Schaffer collaterals 

The Schaffer collateral model describes, in a simplified form, the connections from 
the N CA3 pyramidal cells to the M CAl pyramidal cells. Most Schaffer collateral 
axons project into the stratum radiatum of CAl, although CA3 neurons proxi- 
mal to CAl tend to project into the stratum oriens (Ishizuka ct al., 199C| ); in the 



model these are assumed to have the same effect on the recipient pyramidal cells. 
Inhibitory interneurons are considered to act only as regulators of pyramidal cell 
activity. The perforant path synapses to CAl cells are at this s tage ignored (al- 



though they have been considered elsewhere; see Fulvi-Mari et al. , this volume), as 
are the few CAl recurrent collaterals. The system is considered for the purpose of 
analysis to operate in two distinct modes: storage and retrieval. During storage the 
Schaffer collateral synaptic efficacies are modified using a Hcbbian rule reflecting 
the conjunction of pre- and post-synaptic activity. This modification has a slower 
time-constant than that governing neuronal activity, and thus does not affect the 
current CAl output. During retrieval the Schaffer collaterals relay a pattern of 
neural firing with synaptic efficacies which reflect all previous storage events. In 
the following, the superscript S is used to indicate the storage phase, and R to 
indicate the retrieval phase. 

• {rji} are the firing rates of each cell i of CA3. The probability density of 
finding a given firing pattern is taken to be: 

P({Vi}) = l[Pv(Vi)dVi (1) 

i 

where rj is the vector of the above firing rates. This assumption means that 
each cell in CA3 is taken to code for independent information, an idealised 
version of the idea that by this stage most of the redundancy present in earlier 
representations has been removed. 
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Figure 1 : A schematic diagram of the hippocampal formation. Information enters 
the hippocampus from layer 2 entorhinal cells by the perforant path, which projects 
into dentate gyrus, CA3 and CA1 areas. In addition to its perforant path inputs, 
CA3 receives a lesser number of mossy fibre synapses from the dentate granule 
cells. The axons of the CA3 pyramidal cells project commissurally, recurrently 
within CA3, and also forward to area CA1 by the Schaffer collateral pathway. 
Information leaves the hippocampus via backprojections to the entorhinal cortex 
from CA1 and the subiculum, and also via the fornix to the mammillary bodies and 
anterior nucleus of the thalamus. 
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• {Vi} are the firing rates in the pattern retrieved from CA3, and they are 
taken to reproduce the {r/i} with some Gaussian distortion (noise), followed 
by rectification 



Vi = h + s x } + 



<(^) 2 > 



(2) 



(the rectifying function \xy — x for x > 0, and otherwise, ensures that a 
firing rate is a positive quantity. This results in the probability density of 
Vi having a point component at zero equal to the sub-zero contribution, in 
addition to the smooth component), as can be related (e.g.) to interference 
effects due to the loading of other memory patterns in CA3 (see below and 
Treves and Rolh 1991). This and the following noise terms are all taken to 
have zero means. 

• are t ne firing rates produced in each cell j of CA1, during the storage of 
the CA3 representation; they are determined by the matrix multiplication of 
the pattern {r^} with the synaptic weights Jy - of zero mean, as explained be- 
low, and variance Cj - followed by Gaussian distortion, (inhibition-dependent) 



thresholding and rectification 



6- 



i + 



<«) 2 > 



j' 



(3) 



The synaptic matrix is very sparse as each CA1 cell receives inputs from only 
Cj (of the order of 10 4 ) cells in CA3. The average of Cj across cells is with 
each other, denoted as C 



*i = {0,1} 
( Cij )N = C 3 (C=<C#)) 



(4) 



• {Uj} are the firing rates produced in CA1 by the pattern {Vi} retrieved in 
CA3 



Uj = 

m 2 ) 



-i + 



Un 



J2 c ij J ijVi + e 



'ij , 



a 



J 



(5) 



Each weight of the synaptic matrix during retrieval of a specific pattern, 

4 = cos(0„) 4 + 7 1/2 (^)^(r/ 4 , 0) + sin(^) (6) 

consists of 



1. the original weight during storage, J^, damped by a factor cos{9 fJ ) ) 
where < 6^ < ir/2 parameterises the time elapsed between the storage 
and retrieval of pattern fx (/x is a shorthand for the pattern quadruplet 
fa, V ( , &,£/,•} )• 
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2. the modification due to the storage of [i itself, represented by a Hebbian 
term H(rji,£j), normalised so that 

<(fffa,0) a > = o5; (7) 

7 measures the degree of plasticity, i.e. the mean square contribution of 
the modification induced by one pattern, over the overall variance, across 
time, of the synaptic weight. 

3. the superimposed modifications J N reflecting the successive storage of 
new intervening patterns, again normalised such that 

(8) 

The mean value across all patterns of each synaptic weight is taken to be equal across 
synapses, and is therefore taken into the threshold term. The synaptic weights J-j 
and jfj are thus of zero mean, and variance <Tj (all that affects the calculation is 
the first two moments of their distribution). 

A plasticity model is used which corresponds to gradual decay of memory traces. 
Numbering memory patterns from 1, A, oo backwards, the model sets cos(9\) 
— exp(—Xjo/2) and j(0\) — ^exp{— A70). Thus the strength of older memories 
fades exponentially with the number of intervening memories. The same forgetting 
model is assumed to apply to the CA3 network, and for this network, the maximum 
number of patterns can be stored when the plasticity ■y^ 3 — 2/C 

For the Hebbian term the specific form 

fffai,&) = 4=&-£o) (%-(%)) (9) 

is used, where h ensures the normalisation given in Eq. ^. 

The thresholds £0 an d Uq are assumed to be of fixed value in the following 
analysis. This need not be the case, however, and as far as the model represents (in 
a simplified fashion) the real hippocampus, they might be considered to be tuned 
to constrain the sparseness of activity in CAl in the storage and retrieval phases of 
operation respectively, reflecting inhibitory control of neural activity. 

The block diagram shown in Fig. || illustrates the relationships between the 
variables described in the preceding section. 



Treves, 1995). 



3 Technical comments 



The aim of the analysis is to calculate how much, on average, of the information 
present in the original pattern {77.;} is still present in the effective output of the 
system, the pattern {Uj}, i.e. to average the mutual information 



i({Vi},{Uj}) = 



]Jd Vl Jl[dU 3 P({ Vl }, {U }) In 



P({ Vi},{Uj}) 
})P({Uj}) 



(10) 



over the variables c i:) , J?- 



jN 



The details of the calculation are unfortunately too 
extensive to present here, and the reader will have to be satisfied with an outline 
of the technique used. Those not familiar with replica calculations may refer to the 



final chapter of (Hertz ct al.|, 1991), the appendices of (Rolls and Treves, 1997), or 



less accessibly, the book by [Mezard et al. (1987) for background material. 
P({f]i},{Uj}) is written (simplifying the notation) as 



P(r),U)=P(U\r))P(r)) 



[ [ dVd£P{U 
Jv Ji 



V,S,Ti)P{V\rj)P{£\jj)P{ri) (11) 
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Figure 2: A block diagram illustrating the relationships between the variables 
present in the model. The input of the system could be considered to be the CA3 
pattern during storage, 77, and the output the CAl pattern during retrieval, U. J,^ 
depends on jfj, £j and r\i as described in the text. 



where the probability densities implement the model defined above. 

The average mutual information is evaluated using the replica trick, which 
amounts to 

(P n - 1) 

logP = lim ^ - (12) 



which involves a number of subtleties, for which (Mezard et al., 1987) can be con- 
sulted for a complete discussion. The important thing to note is that an assumption 
regarding replica symmetry is necessitated, and the stability of resulting solutions 
must be checked. This has been reported in (Schultz and Treves, 1997) for a simpli- 
fied version of the neural network analysed here: a single layer of threshold-linear 
neurons with a single phase of operation (transmission) rather than the dual (stor- 
age and retrieval) modes in the model presented here. The conclusions of that study 
were that the replica-symmetric solution is stable for sufficiently sparse codes, but 
that for more distributed codes the solution was unstable below a critical noise 
variance. These conclusions can be assumed to carry across to the current model 
in at least a qualitative sense. In those regions (low noise, distributed codes) where 
the replica-symmetric solution is unstable, a solution with broken replica symmetry 
is required. It should be noted that it is not known what quantitative difference 
such a solution would bring: it may be very little, as is the case with the Hopfield 



network(Amit et al., 1987) 



The expression for mutual information thus becomes 

p(v,uy 



dr]dUP(ri, U) 



P{rf) 



[P{U)] n 



C,J S ,J N 



(13) 

where it is necessary to introduce n + 1 replicas of the variables Si, €j, e^, Vi,£j and, 
for the second term in curly brackets only, r\i. 

The core of the calculation then is the calculation of the probability density 



G 



( x P(r],U) n+l y The key to this is "self-consistent statistics" (Rolls and Treves, 
1997, appendix 4): all possible values of each firing rate in the system are inte- 
grated, subject to a set of constraints that implement the model. The constraints 
are implemented using the integral form of the Dirac (S-function. Another set of 
Lagrange multipliers introduces macroscopic parameters 



N 2^ / v \ v * ^» i 

i v " 

i 



(14) 



where 6{x) is the Heaviside function, and a,f3 are replica indices. Making the as- 
sumption of replica symmetry, and performing the integrals over all microscopic 
parameters, with some algebra an integral expression is obtained for the average 
mutual information per CA3 cell. This integral over the macroscopic parameters 
and their respective Lagrange multipliers is evaluated using the saddle-point ap- 
proximati on, which is exact in the limit of an infinite number of neurons (see, for 



example, Jeffreys and Jeffreys , 1972) to yield the expression given in Appendix A; 



the saddle-points of the expression must in general be found numerically. 



4 How graded is information representation on the 
Schaffer collaterals? 

Specification of the probability density P{rj) allows different distributions of fir- 
ing rates in CA3 to be considered in the analysis. Clearly the distribution of firing 
rates that should be considered in the analysis is that of the firing of CA3 pyramidal 
cells, computed over the time-constant of storage (which we can assume to be the 
time-constant of LTP), during only those periods where biophysical conditions are 
appropriate for learning to occur. Unfortunately this last caveat makes a simple fit 
of the firing-rate distribution from single-unit recordings fairly meaningless unless 
the correct assumptions regarding exactly what these conditions are in-vivo can be 
made. It would be fair to assume that cholinergic modulatory activity is a pre- 
requisite, and unfortunately we cannot know directly from single-unit recordings 
from the hippocampus when the cells recorded from are receiving significant cholin- 
ergic modulation. Note that it might be possible to discover this indirectly. In any 
event, possibly the most useful thing we can do for the present is to assume that the 
distribution of firing rates during storage is graded, sparse, and exponentially tailed. 



This accords with the observations of neurophysiologists (Barnes et al., 1990; Rolls 



md Treves, 1997). The easiest way to introduce this to the current investigation 
is by means of a discrete approximation to the exponential distribution, with extra 
weight given to low firing rates. This allows quantitative investigation of the effects 
of analogue resolution on the information transmission capabilities of the Schaffer 
collateral model. 

The required CA3 firing rate distributions were formed by the mixture of the uni- 
tary distribution and the discretised exponential, using as mixture parameters the 
offset e between their origins, and relative weightings. The distributions were con- 
strained to have first and second moments (77), (rj 2 ) , and thus sparseness (77) 2 / (rj 2 ), 
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equal to a. In the cases considered here, a was allowed values of 0.05, 0.10 and 0.20 
only. The width of the distribution examined was set to 3.0, and the number of 
discretised firing levels contained in this width parameterised as The binary dis- 
tribution was completely specified by this; for distributions with a large number of 
levels, there was some degree of freedom, but its numerical effect on the resulting 
distributions was essentially negligible. Those distributions with a small number 
of levels > 2 were non-unique, and were chosen fairly arbitrarily for the following 
results, as those that had entropies interpolating between the binary and large I 
situations. Some examples of the distributions used are shown in Fig. [|a. 

The total entropy per cell of the CA3 firing pattern, given a probability distri- 
bution characterised by L levels, is 

L 

h(rj) = -J2 p r n (m)^Pr n (m)- (is) 

{=1 

The results are shown in Fig. ||b-d. The entropy present in the CA3 firing 
rate distributions is marked by asterisks. The mutual information conveyed by 
the retrieved pattern of CA1 firing rates, which must be strictly less than the 
CA3 entropy, is represented by circles. It is apparent that maximum information 
efficiency occurs in the binary limit. More remarkably, even in absolute terms 
the information conveyed is maximal for low resolution codes, at least for quite 
sparse codes. The results are qualitatively consistent over sparsenesses a ranging 
from 0.05 to 0.2; obviously with higher a (more distributed codes), entropies are 
greater. For more distributed codes (i.e. with signalling more evenly distributed 
over neuronal firing rates), it appears that there may be some small absolute increase 
in information with the use of analogue signalling levels. 

For comparison, the crosses in the figures show the information stored in CA1. 
This was computed using a simpler version of the calculation, in which the mutual 
information i({rji}, was calculated. Obviously, in this calculation, the CA3 

and CAl retrieval noises as and erf are not present; on the other hand, neither 
is the Schaffer collateral memory term. Since the retrieved CAl information is 
in every case higher than that stored, we can conclude that for the parameters 
considered, the additional Schaffer memory effect outweighs the deleterious effects 
of the retrieval noise distributions. 

It follows from the forgetting model defined by Eq. |^, that information transmis- 
sion is maximal when the plasticity (mean square contribution of the modification 
induced by one pattern) is m atched in the CA3 recurrent collaterals and the Schaf- 
fer collaterals ( Treves , 1995 ). It can be seen in Fig. ||e that this effect is robust to 



the use of more distributed patterns. 



5 Non-uniform Convergence 



It is assumed in ( Treves] , 1995| ) that there is uniform convergence of connections from 
CA3 to CAl across the extent of the CAl subfield. In reality, each CAl pyramidal 
neuron does not receive the same number of connections from CA3: this quantity 
varies across the transverse extent of CAl (although this transverse variance may 
be less than that within CA3; |Amaral et al.| 1990). [Bernard and Wheal| (1994) 



investigated this with a connectivity model constructed by simulating a Phaseolus 
vulgaris leucoagglutinin labelling experiment, matched to the available anatomical 
data. Their conclusion was that mid CAl neurons receive more connections (8000) 
than those in proximal and distal CAl (6500). The precise numbers are not impor- 
tant here; what is of interest is to consider the effect on information transmission 
of this spread in the convergence parameter Cj about its mean C. 
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Figure 3: a Some of the CA3 firing rate distributions used in the analysis. These 
are, in general, formed by the mixture of a unitary distribution and a discrctiscd 
exponential, b - d The mutual information between patterns of firing in CA1 and 
patterns of firing in CA3, expressed in natural units (nats). Asterisks represent 
the entropy of the CA3 pattern distribution, diamonds the CA1 retrieved mutual 
information, and crosses the CA1 information during the storage phase. The hor- 
izontal axis parameterises the number of discrete levels in the input distribution: 
for codes with fine analogue resolution, this is greater, b is for a = 0.05 (sparse), c 
for a = 0.10, and d for a = 0.20 (slightly more distributed), e The dependence of 
information transmission on the degree of plasticity in the Schaffer collaterals, for 
a = 0.05 (solid) and a = 0.10 (dashed). A binary pattern distribution was used in 
this case. 
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Figure 4: Information transmitted as a function of Schaffer collateral plasticity, a 
Binary CA3 firing rate distributions. The solid line indicates the result for the 
realistic convergence model. The dashed lines indicate, in ascending order: (i) 
uniform convergence, (ii) two-tier convergence model with Cj £ {5000, 15000}, (iii) 
two-tier convergence model with Cj € {2000, 18000}. b With more realistic CA3 
firing-rate distributions (the 10-level discrete exponential approximation from the 
previous section). The solid line indicates the result for uniform connectivity, and 
the dashed line the two-tier convergence model with Cj € {5000, 15000}. 



In this analysis Oj is set to 1/C for all cells in the network. C is set using the 
assumption of parabolic dependence of Cj upon transverse extent, on the basis of 
Fig. 5 of (Bernard and Wheal , |1994[ ). In order to facilitate comparison with the 
results reported in [Treves ( [1995 ), C is held at 10,000 for all results in this section. 
The model used (which we will refer to as the 'realistic convergence' model) is thus 
simply a scaled version of that due to Bernard and Wheal, with Cj = 7, 143 at the 
proximal and distal edges of CA1, and Cj = 11,429 at the midpoint. Note that 
this refers to the number of CA3 cells contacting each CA1 cell; each may do so via 
more than one synapse. 

The saddle-point expression ([16]) was evaluated numerically while varying the 
plasticity of the Schaffer connections, to give the relationships shown in Fig. [|a 
between mutual information and 7^ A1 . The information is expressed in the figure 
as a fraction of the information present when the pattern is stored in CA3 (|l5| ) . 

Two phenomena can be seen in the results. The first, as mentioned in the pre- 
vious section (and discussed at more length in Treves, 1995), is that information 
transmission is maximal when the plasticity of the Schaffer collaterals is approxi- 
mately matched with that of the preceding stage of information processing. The 
second phenomenon is the increase in information throughput with spread in the 
convergence about its mean. This is an effect which is not immediately intuitive: it 
means that the increase in mutual information provided by those CA1 neurons with 
a greater number of connections than the mean more than compensates for the de- 
crease in those with less than the mean. It must be remembered that what is being 
computed is the information provided by all CA1 cells about patterns of activity 
in CA3. This increase in information is a network effect that has no counterpart 
in the information a single CA1 cell could convey. In any case, the effect is rather 
small: the realistic convergence model allows the transmission of only marginally 
more information than the uniform model. The uniform convergence approximation 
might be viewed as a reasonable one for future analyses, then. 

Fig. |]b shows that the situation for graded pattern distributions is almost iden- 
tical. The numerical fraction of information transmitted is of course lower (but 
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total transmitted information is similar - see previous section). The uniform and 
two-tier convergence models provide bounds between which the realistic case must 
lie. 



6 Discussion and summary 



This chapter examined quantitatively the effect of analogue coding resolution on 
the total amount of information that can be transmitted in a model of the Schaffer 
collaterals. The tools used were analytical and numerical, and the focus was upon 
relatively sparse codes. What can these results tell us about the actual code used to 
signal information in the mammalian hippocampus? In themselves, of course, they 
can make no definite statement. It could be that there is a very clear maximum for 
information transmission in using binary codes for the Schaffer collaterals, and yet 
external constraints, such as CAl efferent processing, might make it more optimal 
overall to use analogue signalling. So results from a single component study must 
be viewed with duep caution. However, these results can provide a clear picture of 
the operating regime of the Schaffer collaterals, and that is after all a major aim of 
any analytical study. 

The results from this paper reiterate some previously known points, and bring 
out others. For instance, it is very clear from Fig. || that, while nearly all of the 
information in the CA3 distribution can be transmitted using a binary code, this 
information fraction drops off rapidly with analogue level. The total amount of 
information transmitted is similar regardless of the amount of analogue level to be 
signalled - but this is a well known and relatively general fact, and accords with 
common sense intuition. However, the total amount of information that can be 
transmitted is only roughly constant. It appears, from this analysis, that while 
the total transmitted information drops off slightly with analogue level for very 
sparse codes, the maximum moves in the direction of more analogue levels for more 
evenly distributed codes. This provides some impetus for making more precise 
measurements of sparseness of coding in the hippocampus. 

Another issue which this model allows us to address is the expansion ratio of 
the Schaffer collaterals, i.e. the ratio between the numbers of neurons in CAl and 
CA3, M/N. It can be seen in Fig. || that an expansion ratio of 2 (a 'typical' 
biological value) is sufficient for CAl to capture most of the information of CA3, 
and that while the gains for increasing this are diminishing, there is a rapid drop-off 
in information transmission if it is reduced by any significant amount. The actual 
expansion ratio for different mammalian species reported in the literature is subject 
to some variation, with the method of nuclear cell c ounts giving ratios between 1.4 
(Long Evans rats) and 2.0 (humans) ( (5ercss| , 1988 ) , whi le stereological estimates 



range from 1.8 (Wistar rats) to 6.1 (humans) (West, 199C ). It should be noted that 
in all these estimates, and particularly with larger brains, there is considerable error 
(L. Seress, personal communication). However, in all cases the Schaffer collateral 
model appears to operate in a regime in which there is at least the scope for efficient 
transfer of information. 

Clearly it is essential to further constrain the model by fitting the parameters as 
sufficient neurophysiological data becomes available. As more parameters assume 
biologically measured values, the sensible ranges of values that as-yet unmeasured 
parameters can take will become clearer. It will then be possible to address further 
issues such as the quantitative importance of the constraint upon dendritic length 
(i.e. the number of synapses per neuron) upon information processing. 

In summary, we have used techniques for the analysis of neural networks to 
quantitatively investigate the effect of a number of biological issues on information 
transmission by the Schaffer collaterals. We envisage that these techniques, devel- 
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Figure 5: The dependence of information transmission on the expansion ratio 
rcAi,CA3 = M/N. 



oped further and applied in a wider context to networks in the medial temporal lobe, 
will yield considerable insight into the organisation of the mammalian hippocampal 
formation. 



Appendix A. Expression from the Replica Evalua- 
tion 

(i) = exir yA! y A ^Y^ T (yA,w ,z°,C j ,-f)-^y A y A 

+N J D~ Sl (F(s u 0,V,VA,0, 0)lnF(s l5 0,r?, VA,0,0))r, j 

V j 



N 

~-^\VbVb + 2w B w B + z B z B ) 

+N j Ds 1 Ds 2 (F(s 1 ,s 2 ,'n,y B ,w B7 z B )) ri 

x]n(F(h,h,V,yB,WB,z B )) v ^ (16) 

where taking the extremum means evaluating each of the two terms, separately, 
at a saddle-point over the variables indicated. The notation is as follows. N is the 
number of CA3 cells, whereas the sum over j is over M CA1 cells. F is given by 



F(§i,s 2 ,V,y,w,z) = 



x exp 



V + o&{s+ -wrj) 

[y + crj(§+-wr))] 
2*2(1 + 



V 1 + v'iv 



+ < 







~-v~ 


v 2 1 


. °6 _ 


exp ^j 
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x exp 



rjs- 



(17) 



and has to be averaged over P v and over the Gaussian variables of zero mean 
and unit variance si , s 2 . 



Ds 



ds 

7^ 



exp— s 2 /2 <fi(x) 



Ds. 



(18) 



y, w and z are saddle-point parameters. s+ and s_ are linear combinations of $1, s 2 : 
~s± = E(Tl) (fc - 1} 



fe=i 





v /(y-z) 2 + 4w 2 T(-i) fc (y-z) 


(yz — w 2 ) 




y + z + (-l) k ^/(y - z) 2 + Aw 2 


y/{y - z) 2 + Aw 2 



h (19) 



in the last two lines of Eq. [l6[ but in the second line of Eq. |l^ one has s+ = 
hVVA,S- = 0. 

r is effectively an entropy term for the CA1 activity distribution, given by 



dsids 2 , s (T' 7 ) 
exp-( si s 2 

2tt ,/det T' 



si 

s2 



dUG(U)\n I dU'G[U') 

50 J —oc 

dUG{U)hxG(U) , 



(20) 



where 

G(U) = G(U;s 1 ,s 2 ,y,w,z,C j ,j) 



(Co - s 2){T V j + 2gjT wj + g]T zj ) + (U - U + sj + gjS 2 )(T w ,j + gjT ZJ ) 
^{T V] T Z] - T 2 WJ )(T m + 2 9] T W] + g 2 T ZJ ) 
(U -U Q + s 1 +g r s 2 ) 2 



2 A T yj + 2 9jT w] + g]T zj ) 



: exp - 



2 (Tyj + 2gjT w j + g 2 T z j t 



+ 



-(£0 - s 2 )T yj - (U - U + gi + ;'/,, ,. 
{TyjTzj — T 2 -)T y j 



and 



: exp - 



± V3 

T 
T' 



are effective noise terms. 



2T 

^ J. ■>,n 



= a% + a 2 jC 3 (y° - y) 

= (T 2 jCj{w n - iu)cos(0) 

= a 2 s +a 2 Q(z°-z) 

y ui cos(0) 
■u;cos(0) 2 



(21) 



(22) 



C 



g j= h-£x° (V) V VCW), 



(23) 
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y, w, z are saddle-point parameters (conjugated to y, w and z), and x°,y°, w°, z° are 
corresponding single-replica parameters fixed as 




i 

7 Appendix B. Parameter Values 

Parameters used were, except where otherwise indicated in the text: 





0.30 




0.20 


V? 


0.20 


C 


10000 


°*j 


0.0001 


Co 


-0.4 


U 


-0.4 


M/N 


2.0 
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